Variability Analysis of Traffic Simulation Outputs: Practical Approach for TRAF-NETSIM

Stochastic traffic simulation models, such as TRAF-NETSIM, use random number seeds to generate variables to describe driver, roadway, and traffic characteristics. In analyzing outputs from these models, one should consider the variability of the responses. The variability of NETS IM' s output using the methods of replication and batch means was explored. For the batch means method, it is proposed to compute the measures of effectiveness (MOEs) for intermediate time intervals using a proposed interval calculation (PIC) procedure. The MOEs were evaluated at the network, intersection, and link levels of aggregation. Depending on the MOE and level of aggregation, the two methods yielded significantly different results. Hence, depending on the study objective, outputs may need to be examined at different levels of aggregation to obtain meaningful results. The practical implications of the variability are also discussed, and statistical approaches are proposed to deal with output variability. Autoand cross-correlations must be examined explicitly, particularly when dealing with link MOEs resulting from very short simulation time. Ignoring positive crosscorrelation is not detrimental but leads to more conservative confidence intervals. Either the batch means with PIC method or replication method must be used to build confidence intervals. NETS IM' s direct output for intermediate time intervals should not be used to build a confidence interval unless an autocorrelation analysis is done. Not using proper statistical procedures can lead to erroneous and misleading conclusions.